Annotating Content With Context Metadata

ABSTRACT

A method of annotating content with context metadata comprising:
         a) obtaining a first list of context metadata items for a content item;   b) obtaining a second list of context metadata items from the first list, wherein the second list contains some, but not all, of the metadata items of the first list; and   c) annotating the content item with the second list by creating an association between metadata items of the second list and the content item.

FIELD OF THE INVENTION

Embodiments of the present invention relate to annotating content withcontext metadata. In particular, they relate to a method, a device and acomputer program that makes context metadata more meaningful andfacilitates the annotation of content with meaningful metadata.

BACKGROUND TO THE INVENTION

Increasing storage capacity enables users to store very large amounts ofcontent to their mobile phones and other small, portable electronicdevices. While the amount of content that should be managed isincreasing dramatically the physical UI (User Interface) will not changeconsiderably. The display resolutions will increase but the physicaldisplay sizes will probably not increase much since in already existingphones the display covers a large part of the surface area. Text inputspeed is not likely to increase considerably. Other than keyboard-basedtext input, the other various methods (e.g. stylus-based) have notproven to solve the difficulty and slowness of text input and theoverall phone size miniaturization and display size increase needs leaveless and less space for keyboards.

There is a seemingly unbalanced development of the amount of storedcontent and the physical phone user interface (UI).

One solution by Sorvari et al.: “Usability issues in utilizing contextmetadata in content management of mobile devices.” NordiCHI '04:Proceedings of the third Nordic conference on Human-computerinteraction, ACM Press: 357-363 suggests that context metadata may beused to enhance content management.

Context metadata is information that describes the context when contentis created (or received). This context metadata can be used to annotatethe content item. This facilitates efficient content management featuressuch as searching and also facilitates organization features.

However, the context metadata may be ‘contaminated’ with metadata thatis meaningless to a user. For example, if the context metadata includesthe identities of nearby devices, then some of these devices may havemeaning to the user, such as the devices of friends, whereas otherdevices will have no meaning such as the devices of passers by. Thecontamination of meaningful metadata by meaningless metadata may preventthe effective use of contextual metadata.

When a user wishes to annotate a content item with contextual metadata,there may be a problem in selecting meaningful contextual metadata fromthe available contextual metadata. Meaningless contextual metadatacontaminates the meaningful contextual metadata.

Likewise when a user accesses metadata that is stored in associationwith content, there may be a problem in displaying only meaningfulcontextual metadata from the stored contextual metadata.

BRIEF DESCRIPTION OF THE INVENTION

It would be desirable to provide a way of using contextual metadatawithout suffering the problem of contamination.

According to a first embodiment of the invention there is provided amethod of annotating content with context metadata comprising: a)obtaining a first list of context metadata items for a content item; b)obtaining a second list of context metadata items from the first list,wherein the second list contains some, but not all, of the metadataitems of the first list; and c) annotating the content item with thesecond list by creating an association between metadata items of thesecond list and the content item.

According to another embodiment of the invention there is provided anelectronic device for annotating content items with context metadata,the device comprising: a memory for storing a first list of contextmetadata items for a content item and the content item; and a processorfor obtaining a second list of context metadata items from the firstlist, wherein the second list contains some, but not all, of themetadata items of the first list and for annotating the content itemwith the second list by creating associations between metadata items ofthe second list and the content item.

According to another embodiment of the invention there is provided acomputer program for annotating content items with context metadatacomprising computer program instructions which when loaded into aprocessor enable the processor to:

a) obtaining a second list of context metadata items from a first listof context metadata items for a content item, wherein the second listcontains some, but not all, of the metadata items of the first list; andb) annotating the content item with the second list by creating anassociation between metadata items of the second list and the contentitem.

According to another embodiment of the invention there is provided agraphical user interface means in which context metadata items aredisplayed for selection by a user, the user interface means providingmeans for highlighting some of the context metadata items relative toother of the context metadata items to aid user selection of meaningfulmetadata items.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present invention reference will nowbe made by way of example only to the accompanying drawings in which:

FIG. 1 illustrates an arrangement 1 of devices;

FIG. 2 illustrates the relationship between a first list of contextmetadata items and a second list of context metadata items which areused to annotate a content item; and

FIG. 3 schematically illustrates a process of annotating a content itemwith metadata items according to an embodiment of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The Figures illustrate a method of annotating a content item 43 withcontext metadata items 50 comprising:

a) obtaining a first list 41 of context metadata items 50 for a contentitem 43;

b) obtaining a second list 42 of context metadata items 50 from thefirst list 41, wherein the second list 42 contains some, but not all, ofthe metadata items 50 of the first list 41; and

c) annotating the content item with the second list by creating anassociation 44 between metadata items 50 of the second list 42 and thecontent item 43.

Information relating to location, time, data and the proximalenvironment when an action occurs at a user device in relation to acontent item 43 can be used as context metadata for that content item43. The action may be, for example, one of creating, receiving, orediting the content item 43. This metadata makes information availableover and above that present in the content item itself. The contextmetadata may be considered as a collection of metadata items 50, whichprovide different information. Suitable metadata items 50 may include:time, date, location, nearby devices (and for each nearby device itsposition and/or distance from the user device).

The content item 43 is annotated with the context metadata items 50collected when the action occurred at the user device 10 in relation tothe content item 43. This annotation is achieved by creating anassociation 44 between metadata items 50 of the context metadata withthe content item 43. The annotation enables organization of the contentitem 43 based on the context metadata items 50 and/or searching for thecontent item using the context metadata items 50.

Information about the proximal environment to the user device 10 can bedetermined by, for example, receiving identifiers transmitted by localradio devices 2, 4, 6. One type of identifier may indicate a networkarea in which the local device 10 is situated. For example, theidentifier may be a SSID for a WLAN network or a combination of networkID and cell ID for a cellular telecommunications network. Another typeof identifier may indicate the presence of a device nearby. For example,according to the Bluetooth Standard, a Bluetooth (BT) device can performan inquiry procedure that allows it to obtain the BT device IDs and alsothe BT friendly names from the local BT devices. Either the BT device IDor BT friendly name may be used as an identifier. Some BT devices may befixed, in which case the device identifier specifies a location whereasother BT devices may be user portable devices, such as mobile cellulartelephones, which indicate the presence of a particular user.

The context metadata items 50 and particularly the metadata identifyingnearby devices (device identifiers) can be used to considerably enhancethe automatic organization and search capabilities that the user device10 provides. In addition, these metadata items 50 can be also madeavailable in other devices such as PCs to enhance the content managementin other devices.

Automatic detection of context metadata means that in many cases thecontext metadata is “contaminated” with information that is notmeaningful to the user. For example, if the user device captures animage in a public place, it is quite likely that some nearby devices arenot meaningful to the user but will be included in the context metadata.These devices may be owned by strangers that just happened to be nearbywhen the image was captured.

Even with the limited range of present Bluetooth radios it is quitelikely that strangers are within BT range in public places. With WLANtechnology or with some other technology with longer range the number ofmeaningless devices increases in proportion to the square of the range(detection area).

FIG. 1 illustrates an arrangement 1 of electronic devices 10, 2, 4, 6. Auser device 10 is arranged to store context metadata as a first list 41when an action such as creation, receipt, editing occurs with respect toa content item 43. The content item may be, for example, a video clip,an image, an audio file, text, a multimedia presentation etc.

The user device 10 comprises a processor 20, a user input device 22 suchas a keypad, a memory 24, a display 26 and context detection devices 30for obtaining context metadata items. The Figure illustrates only asmany features as are necessary to explain the implementation of theinvention below.

The processor 20 is connected to read from and write to the memory 24,to receive input from the user input device 22 and from the contextdetection devices 30 and to provide an output to the display 26.

The memory 24 stores computer program instructions 25 which when loadedinto the processor 20 enable the user device to operate as described inthe following paragraphs. The computer program instructions provide thelogic and routines that enables the electronic device 10 to perform themethods illustrated in FIGS. 2 and 3.

The computer program instructions may arrive at the electronic device 10via an electromagnetic carrier signal or be copied from a physicalentity such as a computer program product, a memory device or a recordmedium such as a CD-ROM or DVD.

The metadata items 50 and content items 43 are stored in a database 27in the memory 24. This database also stores information about theassociations between metadata items 50 and content items 43.

The context detection devices 30 may include a time/date detector 32such as a real time clock, a location detector 34 such as a GPS receiverand a proximity detector 36 that detects devices 2, 4, 6 that areproximal to the user device 10. The proximity detector may be aBluetooth device which communicates with Bluetooth devices 2, 4, 6 or aWLAN, WiMAX or UWB device. Alternatively some other proximity radiotechnology can be utilized to detect nearby devices. In addition, it mayalso be possible to detect nearby devices if the devices share theirlocation over a (wireless) network.

The context detection devices 30 provide context metadata items to theprocessor 20. These may include: time, date, location, nearby devices(and for each nearby device its position and/or distance from the userdevice).

The process of the invention is schematically illustrated in FIGS. 2 and3.

FIG. 2 illustrates the relationship between a first list 41 of contextmetadata items 50, which are originally obtained when an action isperformed in relation to a content item 43 and a second list 42 ofcontext metadata items which are used to annotate 44 the content item43. The first list 41 of context metadata items 50 includescontamination, that is, context metadata items 50 that are meaninglessto the user of the user device 10. The second list 42 of metadata items50 comprises a sub-set of the first list 41 and includes those metadataitems 50 in the first list 41 that are meaningful. A filterautomatically filters the first list 41 of metadata items to identifymetadata items in the first list that are probably meaningful and/or toextract automatically metadata items in the first list that are probablymeaningful to the second list 42. The content item 43 is annotated byassociating 44 each of the metadata items 50 of the second list 42 withthe content item 43.

FIG. 3 schematically illustrates the process of annotating a contentitem with metadata items according to an embodiment of the invention.

At step 60, an action is performed on a content item 43. The action maybe, for example, creation, receipt or editing of the content items 43.The content item 43 is, for example, a video clip, an image, an audiofile, text etc

In responses to step 60, at step 62 a first list 41 of context metadataitems 50 is obtained for the content item 43. It is stored inassociation with that content item in the database 27. If the contentitem is received at the user device 10, then the context metadata itemsof the first list 41 may be context metadata items received with thecontent item 43 and/or context metadata items determined by the contextdetector devices 30 of the user device 10 on receipt of the content item43. If the content item 43 is created, then the context metadata itemsof the first list 41 are those determined by the context detectiondevices 30 of the user device 10 on creation of the content item 43.

After step 62, at step 64, a second list 42 of context metadata items isobtained from the first list. This is discussed in more detail below. Itis stored in association with the content item in the database 27.

Then at step 66, the content item 43 is annotated with the second list42 by creating an association between the metadata items 50 of thesecond list and the content item 43.

The step 64 of obtaining a second list 42 of context metadata items 50from the first list 41 involves filtering of the first list to improvethe meaningfulness of the metadata items.

The step 64 may involve a manual step, in which case the first list 41is displayed, the filter is used to highlight some metadata items overothers, and the user selects at least some metadata items from firstlist 41 for use in the second list 42. The highlighting may be achievedby using a visible score for each metadata item and/or by prioritizingthe metadata items according to a score 51. The manual transfer can beperformed when the action occurs or at some time afterwards. Thepresentation is provided by a graphical user interface means in whichthe context metadata items are displayed on display means 26 forselection by the user using the input means 22, the processor 20 andmemory 24 together providing means for the highlighting some of thecontext metadata items relative to other of the context metadata itemsto aid user selection of meaningful metadata items.

The step 64 may involve an automated step, in which case, the first list41 may be but is not necessarily displayed, the filter selects at leastsome metadata items from first list 41 for use in the second list 42.The filter may use a score 51 for the selection.

The score 51 for a metadata item 50 may depend upon a history of the useof that metadata item for annotating content items 43. For example, thefiltering may link a subject metadata item of the first list 41 tocontent items which it already annotates. This may be achieved byaccessing previously stored second lists 42 of metadata in the database27, and identifying those second lists 42 of metadata in which thesubject metadata item of the first list 41 occurs.

Each metadata item 50 may include a score field 51 that identifies, inthis example, the number of content items with which that metadata item50 is already associated. This score field 51 introduces meaningfulnessto metadata items 50 by identifying those metadata items the userstatistically prefers to use for annotating content items 43. Thisallows the user to recognize metadata items that are probably morelikely to be used from the first list of metadata items for annotation.The metadata items of the first list may be ordered in descending orderaccording to the value of the field 51.

The score 51 for a metadata item 50 may depend upon a history of theappearance of that metadata item in stored first lists 41 that areassociated with different content items. This may be achieved byaccessing previously stored first lists 41 of metadata in the database27, and identifying those stored first lists 41 of metadata in which thesubject metadata item of the present first list occurs. Each metadataitem 50 may include a score field 51 that identifies, in this example,the number of first lists 41 on which that metadata item 50 appears.

The score field 51 may then be used as a criterion by a user formanually transferring metadata items 50 from the first list 41 to thesecond list 42. Alternatively or in addition, the score field 51 may beused as a criterion for automatically transferring metadata items 50from the first list 41 to the second list 42 without user intervention.A metadata item 50 is automatically added to the second list 42 if thescore field 51 for that metadata item exceeds a threshold value. Thethreshold value may be different for different content item types. Thatis the threshold depends upon whether the content item with which thesecond list of metadata items is to be associated is audio, text, image,video etc. Alternatively or in addition there can be a compositethreshold for all or selected groups of content item types.

The score field 51 is especially important for metadata items thatrepresent the identifiers of nearby devices 2,4,6, as the identifier ofa device is not always meaningful to a user. When a manual transfer of adevice identifier occurs the user may be given an opportunity to amendor augment the content of that metadata item to make it more meaningfulor to associate the device identifier with an existingalias/identity/contact or other object describing the device or itsowner. Such object can be available e.g. in phonebook, (instant)messaging, publishing or presence application.

The score field 51 may be calculated in other ways as an addition or asan alternative to the way described above. The score field may be madedependent on other attributes than just the number ofassociations/annotations.

The score may be dependent on, for example, any one or a combination ofthe following attributes:

a) Number of associations 44 between the metadata item and other contentitems 43. The larger the number of associations the greater the score 51and the smaller the number of associations the smaller the score 51. Theassociations may occur as a result of the appearance of the metadataitem in a plurality of first lists 41. Alternatively, the associationsmay occur as a result of the appearance of the metadata item in aplurality of second lists 42.

b) How recently associations 44 between the metadata item and othercontent items have been made. More weight is given to more recentassociations compared to older associations. The associations may occuras a result of the appearance of the metadata item in a plurality offirst lists 41. Alternatively, the associations may occur as a result ofthe appearance of the metadata item in a plurality of second lists 42.

c) The type of action that occurs for a content item and/or the contenttype E.g. capturing an image/video, recording a sound clip, receiving amessage, editing, etc.

d) For a device identifier, a distance and/or direction of the devicerepresented by the device identifier relative to the user's device whenthe action occurs for the content item. If the distance is large or ifthe device is behind the user, when the action is capturingimages/videos, the weighting of the score will be low. However, if thedistance is very small and/or the device is in front of the user, whenthe action is capturing images/videos, the weighting of the score willbe high. To implement this option, the first list of metadata items willinclude for each nearby device metadata that provides the deviceidentifier of the nearby device and also metadata that provides thedistance and/or direction to that nearby device.

The local environment of the device from which content metadata item isobtained may also be used in score calculation. The environment mayinclude the number of devices, distance and/or direction, score fornearby devices.

As an example, for a device identifier, the number of devices, distanceand/or direction of other devices nearby when the action occurs for thecontent item. These attributes can be used to detect whether the devicethat provided the device identifier is in the middle of a big crowd inwhich case the score is given a low weighting or whether the device isone of a few devices in which case the score is given a greaterweighting. To implement this option, the first list of metadata itemswill include for each nearby device metadata that provides the deviceidentifier of the nearby device and also metadata that provides thedistance and/or direction to that nearby device.

As a further example, for a device identifier, the number of devices,distance and/or direction of high scoring devices nearby the devicerepresented by that device identifier when the action occurs for thecontent item. High scoring means any device whose device identifier hasalready been added to the second list or whose device identifier has ahigh score field. If a device is detected in the middle of or near highscoring devices, its score can be weighted highly.

More complex mechanisms for determining a score may additionally oralternatively be used. For example, the score associated with a deviceidentifier may be dependent upon the score the device represented bythat device identifier has for the user device 10. For each deviceidentifier in the first list 41, the user device sends a score requestmessage to the device represented by that device identifier (targetdevice). The score message includes the device identifier of the user'sdevice. The target device either retrieves the score associated with thereceived device identifier or calculates a score for that deviceidentifier. It then replies with a score reply message to the user'sdevice which includes the calculated score. The user's device weightsthe score for a device identifier in dependence upon the score receivedin the score reply message received from the respective device. Thehigher the score received, the higher the weighting applied to the scoreof the device identifier.

Thus if the user's device is considered meaningful by another nearbydevice in the first list 41 that other device may be added to the secondlist 42 which includes the meaningful devices. Thus devices thatconsider you meaningful may be added to your second list of meaningfuldevices.

As a further option, the score reply message may additionally oralternatively include the identities of the highest scoring deviceidentifiers of the target device. The device identifiers present in thefirst list that are also present in such a score reply message areautomatically transferred to the second list or their score is heavilyweighted. A further metadata item, a marker, may be associated with adevice identifier that is added to the second list in this way. Thismarker is also transferred in the score reply message and is increasedby one each time it is received. The device identifier score received ina score reply message at a user device is only valid for increasing thescore of that device identifier if the marker is less than a predefinedthreshold.

An augmentation process may also occur on the second list at step 64.This process adds user recognizable meaning to apparently meaninglessmetadata items in the second list.

For example, instead of or in addition to giving a co-ordinate for alocation the coordinate may be represented as a region name. The regionname may correspond to the smallest geographic region previously used bythe user device that is accurate for this location. Alternatively, oradditionally, the location may be compared against the locations ofaddresses stored within a Contacts database. The location may then begiven as, for example, ‘near John Doe's home’, if John Doe's homeaddress is the nearest address in the Contacts database and is within athreshold distance.

For example, instead of or in addition to giving a date and time, thedate and time may be represented as period relative to an entry in acalendar application. If the date and time corresponds to a meeting withJohn, then the user device may additional provide as metadata ‘Meetingwith John at his office’ or if the date and time follows the meeting,the user device may provide the additional metadata ‘after meeting withJohn at his office’

The different options for additional metadata may be presented forselection by the user.

The filtering results in a second list of metadata items which aremeaningful to the user device. The second list may be presented to theuser. The augmentation process may further improve the meaningfulness ofthe metadata items presented to the user.

An organizer application can be used to organize/manage the storedcontent items 43 using their respective second lists 42 of metadataitems 50. A search facility may be based upon the metadata items in thesecond lists 42.

Although embodiments of the present invention have been described in thepreceding paragraphs with reference to various examples, it should beappreciated that modifications to the examples given can be made withoutdeparting from the scope of the invention as claimed.

Whilst endeavoring in the foregoing specification to draw attention tothose features of the invention believed to be of particular importanceit should be understood that the Applicant claims protection in respectof any patentable feature or combination of features hereinbeforereferred to and/or shown in the drawings whether or not particularemphasis has been placed thereon.

1. A method of annotating content with context metadata comprising: a)obtaining a first list of context metadata items for a content item; b)obtaining a second list of context metadata items from the first list,wherein the second list contains some, but not all, of the metadataitems of the first list; and c) annotating the content item with thesecond list by creating an association between metadata items of thesecond list and the content item.
 2. A method as claimed in claim 1,wherein obtaining the second list of context metadata items from thefirst list involves processing the first list to improve themeaningfulness of the metadata items.
 3. A method as claimed in claim 2,wherein the processing involves the addition of metadata.
 4. A method asclaimed in claim 3, wherein the additional metadata is dependent uponpreviously stored data.
 5. A method as claimed in claim 4, wherein theadditional metadata comprises a score.
 6. A method as claimed in claim4, wherein the additional metadata comprises user meaningful informationbased on data obtained from a personal database.
 7. A method as claimedin claim 1, further comprising: displaying the first list; highlightingsome metadata items in the first list compared to other metadata itemsin the first list; and and enabling user selection of at least somemetadata items from first list for use in the second list.
 8. A methodas claimed in claim 7, wherein highlighting may be achieved by usingvisible scores for metadata items.
 9. A method as claimed in claim 7,wherein highlighting may be achieved by prioritizing the metadata itemsaccording to a score.
 10. A method as claimed in claim 1 whereinprocessing automatically selects at least some metadata items from firstlist for use in the second list.
 11. A method as claimed in claim 10,wherein metadata items in the first list are scored and the scores areused as a selection criterion.
 12. A method as claimed in claim 1,wherein obtaining the second list of context metadata items from thefirst list involves scoring of the metadata items of the first list. 13.A method as claimed in claim 5, wherein a score for a metadata itemdepends upon a history of the use of that metadata item for annotatingcontent items.
 14. A method as claimed in claim 13, wherein the scorefor a metadata item depends on the number of times that metadata itemshas previously been used for annotating content items.
 15. A method asclaimed in claim 13, wherein the score for a metadata item depends onhow recently that metadata item has been used for annotating contentitems.
 16. A method as claimed in claim 5, wherein the score for ametadata item that has been received from a device depends upon a localenvironment of that device.
 17. A method as claimed in claim 5, whereinthe score for a metadata item that has been received from a devicedepends upon the score that device has for a metadata item correspondingto another device.
 18. A method as claimed in claim 1 wherein the secondlist may include metadata items not present in the first list.
 19. Anelectronic device for annotating content items with context metadata,the device comprising: a memory for storing a first list of contextmetadata items for a content item and the content item; and a processorfor obtaining a second list of context metadata items from the firstlist, wherein the second list contains some, but not all, of themetadata items of the first list and for annotating the content itemwith the second list by creating associations between metadata items ofthe second list and the content item.
 20. An electronic device asclaimed in claim 19, wherein the device is hand-portable.
 21. A computerprogram for annotating content items with context metadata comprisingcomputer program instructions which when loaded into a processor enablethe processor to: a) obtaining a second list of context metadata itemsfrom a first list of context metadata items for a content item, whereinthe second list contains some, but not all, of the metadata items of thefirst list; and b) annotating the content item with the second list bycreating an association between metadata items of the second list andthe content item.
 22. A computer program as claimed in claim 21, whereinobtaining the second list of context metadata items from the first listinvolves processing the first list to improve the meaningfulness of themetadata items.
 23. A computer program as claimed in claim 22, whereinthe processing involves the addition of metadata.
 24. A computer programas claimed in claim 23, wherein the additional metadata is dependentupon previously stored data.
 25. A computer program as claimed in claim24, wherein the additional metadata comprises a score.
 26. A computerprogram as claimed in claim 21, wherein the additional metadatacomprises user meaningful information based on data obtained from apersonal database.
 27. A computer program as claimed in claim 21,further comprising computer program instructions for: displaying thefirst list, highlighting some metadata items in the first list comparedto other metadata items in the first list; and and enabling userselection of at least some metadata items from first list for use in thesecond list.
 28. A computer program as claimed in claim 21 forautomatically selecting at least some metadata items from first list foruse in the second list.
 29. A computer readable medium having a computerprogram stored thereon for performing the method of claim 1 whenexecuted on a processor.
 30. A physical entity embodying a computerprogram for carrying out the method as claimed in claim
 1. 31. Agraphical user interface means in which context metadata items aredisplayed for selection by a user, the user interface means providingmeans for highlighting some of the context metadata items relative toother of the context metadata items to aid user selection of meaningfulmetadata items.